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THE INCREASED USE OF PROGRAMED MATERIALS CREATE A NEED 
TO ESTABLISH PROCEDURES FOR DETERMINING THE READING 
DIFFICULTY OF THE MATERIALS BEFORE EXTENSIVE FIELD TESTING 
BEGINS. AN EXPERIMENT WAS CARRIED OUT WITH 169 FOURTH GRADEi 
271 FIFTH GRADEi AND 140 SIXTH GRADE STUDENTS. THE POPULATION 
OF MATERIALS FOR THIS STUDY CONSISTED OF NINE FOURTH GRADE 
PROGRAMED SCIENCE UNITS. THE DEPENDENT VARIABLE WAS DEFINED 
AS THE AVERAGE NUMBER OF ERRORS THAT EACH STUDENT MADE IN 
THREE SELECTED PROGRAMED SCIENCE UNITS. SEVENTEEN INDEPENDENT 
VARIABLES WERE CHOSEN AS TO REPRESENT FUNCTIONAL AND 
GRAMMATIC ASPECTS OF THE UNITS. REGRESSION ANALYSIS WAS 
UNDERTAKEN TO DETERMINE THE TOTAL AND RELATIVE INFLUENCE OF 
THE INDEPENDENT VARIABLES ON ERROR COUNT. THE ERROR COUNT WAS 
FOUND VALID AS A CRITERION FOR DETERMINING READING DIFFICULTY 
BECAUSE (1) CORRELATIONS WITH THE RESULTS OF THE UNIT TESTS 
WERE ALL SIGNIFICANT IN PREDICTED DIRECTION! AND (2) BECAUSE 
THREE INDEPENDENT VARIABLES WERE SIGNIFICANT AS PREDICTORS OF 
VARIATIONS IN ERROR COUNT. IN FUTURE STUDlESi IT IS 
RECOMMENDED THAT INDEPENDENT VARIABLES BE GIVEN AMPLE 
CONSIDERATION! AND THAT SAMPLE BE ENLARGED TO INCLUDE LARGER 
NUMBER OF UNITS. (OH) 
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Determining the reading difficulty of books is not a re- 
cent development, but only in the last few years had the attempt 
been made to examine the readability of books by scientific 
procedures* i*any types of materials have been examined but 
little work has been done with the readability of programmed 
materials. Yet indications are that programmed materials will 
be of increasing importance; thereof oi\% there is a need to establish 
procedures for determining the reading difficulty of the materials 
before extensive field testing begins. 



STATbbbiirr OF WH PROBLtiFi 

The purpose of this study was to automate a procedure 
which could be used to aid in determining the reading difficulty 
of selected proerrammed materials. The study involved: (i) the 

development of a regression equation which could be used to 
predict the reading difficulty of the given materials; (2) the 
selection of samples of various sizes and types to determine 
under which sampling procedure the best predictions of readability 
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could be made in relation to the total samples and (3) the 
evaluation of the sequence of the materials to determine if the 
assumptions of good sequential development were satisfied o 

TECHNIQUES AND PROCEDURES 

In order to achieve the purposes of this study « the 
following steps had to be employed : 

io Independent variables appropriate for programmed 
materials were selected « 

2o Computer programs were developed which count 
all of the independent variables automatically# 

3# The criterion variable was defined as the average 

number of errors that each student made in the units* 
Regression equations were determined for the 
criterion and Independent variables* 

5* Different sampling techniques were compared with 
the total sample effectiveness* 

Selection of toterials 

The population of materials for this study consisted of 
nine fourth grade programmed science units* From the nine units 
three were selected to be Included in the sample: (1) sound* 

(2) light* and (3) heat* The three units under investigation 
were constructed * linear programmed units which had been tested 
and revised. 
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Selection of the Students 

The programmed materials were used In twenty elementary 
svhools In Central Vlrfrlnla during the school year 1966-6?. 

Ths population of students who studied science by means of the 
programmed science materials consisted of 140 sixth graders, 271 
fifth graders, and l89 fourth graders. 

Determining the Variables 
lirror Count for the Criterion Variable 

To establish a criterion variable which could be obtained 
from the completed units and still be a measure of reading 
difficulty, an error count was made. The number of errors 
was obtained for each individual per sample of material, and the 
averuge number of errors was calculated « The error count was 
defined as the relative difficulty of a frame. 

The Independent Variables 

i’rom an analysis of the previous readability studies and 
the ma*;erlals under Investigation, the following variables were 

determined for each sample: 

Average number of paragraphs per frame. 

Average number of sentences per paragraph. 

Average number of words per sentence. 

Average number of letters per word. 

X-: Average number of simple sentcmces. 

5 

Average number of woras which were outside Thorndike’s 
” list of 6000 words (heasure of difficult words). 
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Average number of mathematical and scientific words© 

Xo! Average number of mathematical and scientific numerals 
or symbols© 

Xq: Per cent of frames that were response frames per sample© 

Xiqs Per cent of response frames that were structured 
^ response frames© 

X : Per cent of response frames that were free response 

frames© 

X- 2 >: Per cent of frames thSit were non-response frames© 

X^«; Per cent of responses requiring mathemtlcal or 
scientific words© 

X Average number of frames using same key word or 
phrase consecutively © 

X : Average number of words in per average 

number of words in phrases in succeeding frames 
(measure of redundancy)© 

Xj^^: Average number of disjoint frames© 

^17’ Average number of review frames© 

The Regression Analysis 

The counts of the variables for the total sample were used 
to obtain, by use of a computer, a regression equation which 
contained all of the variables© Using the i*" -ratio and the multiple 
R, a procedure was established for systematic deletion of variables© 
Regression equations were produced which contained the significant 
variables© A null hypothesis for each variable was either rejected 
or accepted at the 0,01 level of significance© A null hypothesis 
for the significance of the final equation was also determined , 
and an analysis of variance procedure was used to determine the 
status of this null hypothesis© 
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Sampling: Procedures 

Once the proper deletions had been made three general 
sampllnff procedures were used to determine If and when a 
precision index could be reached for the Independent variables. 
Two random samples were used , one with n ss 50 pages and one with 
n * 97 pages. (Jsing these samples, the established procedure 
of the regression analysis was repeated and equations produced. 

A similar technique was used with the two sequential samples. 

The first one-third of the total sample was used and then, the 
first two-thirds of the total sample was used. This sequential 
procedure was continued by taking the first one-third of each 
unit and putting these parts together* A cluster-sequential 
sample was the result* Two thirds of each unit were then grouped 
together for the final cluster-sequential sample* A null 
hypothesis with which to Judge the significance of difference was 
stated for each different sampling procedure and the total sample, 
and between each sampling procedure* 

DfiSIGN AND ANALYSIS OF DATA 

Phe number of errors made by the students who used the 
programmed science materials during the school year 1966-67 was 
obtained. The average number of errors made was used to measure 
the reading difficulty of the frames. The validation of the 
criterion variable produced negative correlation coefficients, as 
predicted, because the assumption had been made that the larger a 
student^s technical vocabulary, the fewer errors he would make 



o 



6 



in the proccram. The correlation coefficients were all significant 
at the 0*01 level. See Appendix I for a table of results. 

Total Sample 

The total sample for this study consisted of three units: 

11) Sound, (2) Ll;;ht, and (3) Heat. Errors were counted for the 
students who completed these three units and took the unit tests. 

The Computer Program 

The selection of variables, the counting of the variables, 
and the use of these variables have been composite problems In 
past studies of readability. This particular section was designed 
to eliminate the counting problem. A computer program was con- 
structed into which could be read the entire units; listings and 
counts were the result. This program seized the n»Jor purpose of 
using alphabetic characters and numerical characters simultaneously. 
Generally, the program Involved sorting, locating, and accumulating 
the data. The data were the context and responses of the three 
programmed units. The counts obtained by using this program can 
be seen in Appendix II. 

Besides this table the program produced two listings of words, 
one for context and one for the responses. All words in each list 
were alphabetized and the page and frame on that page in which the 
word appeared were listed. 

Basic Data 

The data for the regression analysis consisted of the counts 
on the variables for the 188 pages of the three units. The data 
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were obtained by usincc the computei' and the results of the 
basic pros?ram. 
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degression Analysis 

To determine which Independent variables played significant 
roles In predicting the criterion variable a stepwise multiple 
regression technique was used. In this process the variables were 
entered into the program In order of most significant to least 
slgnlf leant • 

Since a major objective of this Investigation was to study 
the effects of the selected Independent variables on the criterion 
variable* it was necessary to control the effects of the extraneous 
variables* A multiple regression technique was used in which 
full and restricted mathematical regression models \sere defined. 

The full model represents an attempt to express the dependent 
variable ® linear combination of all the other variables. 

The restricted models reflected the effects on the predictive 
Information of factors considered In various forms. The purposes 
for the restricted models In this study were to test a null 
hypothesis for each variable and various combinations of variables. 

Hypotheses 

The form of the hypotheses for the variables that were 
stated was as follows: 

X. (where i * i, 2, 3. •<>• # 17) does not contribute 
to the*regresslon equation to predict the reading difficulty 
as determined by average number of errors per sample (a^ * 0 ) • 

A null hypothesis was determined to test the significance 

of the final rewgresslon equation. 
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Regression Analysis Using Different Types of Samples 

As a minor purpose of this study limited examinations were 
made as to size and type of sample which would enable valid 
predictions • and comparisons were made on means and standard 
deviations of the variables to determine those values which 
were the same to two significant numbers o The multiple R*s 
were the only statistics subjected to testlngo Appendix III 
contains the precision Indexes for each variable to which each 
sample was compared o The full model and the basic restricted 
models remained the same as established with the total sample o 
Sequential sampling was used: (l) one^thlrd of total and (2) two- 

thirds of the total « Cluster-sequential sampling was used: 

(1) one-third of each unit under Investigation and (2) two- 
thirds of each unit under Investigation* Random sampling was 
used: (l) a random sample of fifty pages and (2) a random sample 

of ninety f-seven pages. 

FINDINGS, CONCLUSIONS, AND RECOMltSNDATIONS 

Pindlngs 

From an analysis of the literature and an examination of 
the programmed materials seventeen Independent variables were 
selected. 

The error count, or dependent variable, was done using five 
measures of achievement and one IQ measure. All correlations were 
negative and significant at the O.Oi level of significance (see 
Appendix IV). 
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Only hypotheses conceding variables seven, nine, seventeen, 

and eighteen were rejected; therefore the multiple regression 

equation which gave the best prediction of Y was: 

Y « 0.02129989JC« + 0,00217258Xq + 

0,05553026X^7 - 0,06129922^ 

This equation was used to test hypothesis that tt * 0 and , since 
the ?-ratlo was significant, the null hypothesis was rejected. 

After the establishment of the regression equation and 
procedures for full models and restricted models, six different 
samples were taken and the general null hypotheses were tested: 

1, There will be no significant difference between the 

multiple correlations obtained from: (l) random 

sampling, (2) sequential sampling, and (3) cluster- 
sequential sampling and the total sampling procedure, 

2, There will be no slgnlflcar.t difference between the 

multiple correlations of each nontotal sampling 
procedures: (l) random sampling, (2) sequential 

sampling, and (3) cluster-sequential sampling. 

Hypothesis one was accepted at the 0,05 level of significance and 

hypothesis two was rejected at the same level. 



Conclusions 

As a result of this study It was concluded that the equation 
above Is the best predictor of the reading difficulty of the 

materials under Investigation, 

The error count appears to have validity as a criterion 
variable In determining reading difficulty In programmed materials 
because; (l) the correlations between the tests and the average 
number of errors were all significant and negative, and (2) the 
three independent variables above, were significant as predictors 
of the average number of errors and are characteristic of programmed 
naterlals, with X7 serving a significant role in past formulas of 

readability o. 
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It was concluded that under the conditions and assumptions 
of this study the three significant variables, (the average 
number of mathematical and scientific words (terms) per sample), 

Xq (the per cent of frames that were response frames per sample), 
and X, « (the average number of review frames per sample), were 
considered to be those variables that best predict the reading 
difficulty of the programmed science materials (see Appendix 

IV) o 

The materials did satisfy the assumptions of good sequential 
development: (l) as students progressed through the units their 

number of errors decreased,, (2) the majority of the frames had a 
degree of overlap and the disjoint frames were, to a large degree, 
review frames, (3) frames were written from simple to complex as 
the number of mathematical and scientific terms did increase the 
number of frames increased, Ja.nd this was one of the significant 

variables , 

It was further concluded that, with the exception of the 
total sample, the different sampling procedures used In this 
study did not serve to establish the same level of precision 
for all of the variables as with the total sample o Therefore, for 
this study, one partial sampling procedure could be rated as good 
as any other o However, some doubt was raised concerning small 
random samples as significant H»s occurred between the random sample 
of fifty and the cluster* sequential samples. 

L^rom the above information It was concluded that, until 
different and more exact sampling procedures are developed, the 
total sample should be used, and the only feasible way to obtain 
a total sample Is by use of a computer. 
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aeoommendations 

Por future studies concerning programmed materials* It Is 
recommended that Independent variables specific to programmed 
materials be given ample consideration* end that the computer 
program be extended so that* once the materials has been read In* 
the significant regression equation can be obtained without 
manual manipulation of variables o 

It Is also recommended that, as a future study* the sample 
be enlarged to Included a larger number of units than was used 
In this study* and that the sampling procedures be broken down 
Into smaller proportions to determine If and when a variable 
stabilized before the total sample* 

From the conclusions of this study* It seems reasonable to 
recommend that programmed materials subjected to this type of 
analysis should be analyzed In terms of the concepts and skills 
to be taught by the program. 
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